Epipolar Constraints for Multiscale Matching

Many recent keypoint detectors associate a local scale (for multiscale detectors) or even a full affine frame (for affine-invariant detectors) to each detected keypoint. Although conventional epipolar constraints are a powerful tool for matching point-like features between pairs of images, they provide no constraint on their relative scales. We present an enhanced epipolar constraint that exploits these scales, thus providing more accurate correspondence search. The method encodes multiscale keypoints as image ellipses, invokes the classical Kruppa constraints that link corresponding ellipses, reduces these to constraints on 1-D quadratic forms on the pencil (1-D family) of epipolar lines, and enforces a scale-sensitive error model by a well-chosen algebraic transformation of the resulting homogeneous representation. The required projections onto the epipolar pencil are extracted from the Singular Value Decomposition of the Fundamental matrix. The final method is very simple to use. Illustrative tests yielded 2–4 fold reductions in false matches for both synthetic and real images. Matlab code is available.